REMARKS 

Preliminary to further examination on the merits, Applicants respectfully request 
entry of the foregoing amendments and the following remarks. 

Claims 122-170 are pending in the present application, with Claims 122-124, 141- 
143, 168 and 170 being independent. Claims 98-121 are cancelled herein without prejudice to or 
disclaimer of the subject matter contained therein. Claims 122-170 are newly presented. No new 
matter is believed to have been added. 

In the Office Action of August 14, 2003, the drawings were objected to for failing 
to clearly show features of the subject matter sought to be patented. Since the claims noted in the 
Office Action as forming the basis for the objection have been cancelled, Applicants submit that 
the objection to the drawings is moot. Further, Applicants believe that the subject matter of the 
now-pending claims is shown in the drawings. 

The title was objected to as not being sufficiently descriptive of the invention. 
The present title was suggested by the Examiner in the December 13, 2002 Office Action, when 
Claims 34-83 were pending. Since new Claims 122-160 are based on many of Claims 34-83, 
reconsideration and withdrawal of the objection to the title are requested. 

The specification was objected to as failing to provide proper antecedent basis for 
the subject matter in Claim 99. Since Claim 99 has been cancelled herein, Applicants, without 
conceding the propriety of the objection, respectfully submit that the objection to the 
specification is moot. 

Claims 98-1 16 and 118-121 were rejected under 35 U.S.C. § 101 as being non- 
statutory subject matter. Claims 98-103, 108-1 13, and 1 17-120 were rejected under 35 U.S.C. § 
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103(a) as being unpatentable over "Towards a Formal Framework for Linguistic Annotations" 
("Bird"). Claims 115 and 121 were rejected under 35 U.S.C. § 102() as being unpatentable over 
Bird in view of "A Fast Lattice-Based Approach to Vocabulary Independent Wordspotting" 
("James"), and in view of U.S. Patent No. 4,227,176 ("Moshier"). Since Claims 98-121 have 
been cancelled, these rejections are submitted to be moot. 

In the Office Action dated December 13, 2002, the Examiner rejected several of 
Claims 31-83 as being unpatentable over Bird in view of U.S. Patent No. 5,500,920 ("Kupiec"). 
Since new Claims 122-160 are based on some of these claims, Applicants submit the following 
comments. 

Bird describes, in section 3, a number of existing annotation systems. Of these, 
the TIMIT, PARTITUR and EMU annotations include word and phoneme data. However, all of 
these databases are created by trained linguistic experts who listen to audio and manually 
annotate audio data with words and phonemes. Applicants submit that the "software" to which 
Bird refers in section 1 relates to a graphical user interface that allows the expert linguist to listen 
to the audio and to mark up the audio with the corresponding words and phonemes and not to 
automatic speech recognition software as suggested by the Examiner. Further, these databases 
are used for the evaluation of automatic speech recognition systems (as acknowledged for the 
TIMIT database discussed in section 3.1). It would therefore not make sense to generate these 
databases using an automatic speech recognition system since the databases would then be 
useless for their intended purpose. In particular, it is only because an expert linguist has 
annotated the speech that it can be used to evaluate different automatic speech recognition 
systems. Applicants therefore submit that it would not have been obvious to generate the 
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annotation data described by Bird using an automatic speech recognition system. It is only with 
hindsight, based on knowledge of the present invention, that this assessment could be made. 

Kupiec describes an information retrieval system which includes a database of 
natural language text which is to be searched. The system described by Kupiec allows users to 
input spoken queries which are processed by a speech recognizer to generate a sequence of 
phonemes representing a spoken utterance. This phone sequence is then processed by a 
hypothesis generator which generates alternative phone sequences in order to take into account 
the possible errors introduced by the speech recognizer. The Kupiec system then compares these 
phone sequences with the phone sequences defined in a phonetic index to identify words 
corresponding to each of the phone sequences. The thus-identified words are then used to search 
for matches in the text database for information to be retrieved. 

With regard to the discussion and interpretation of Kupiec in Section 75 of the 
December 13, 2002 Official Action, Applicants comment as follows. 

The Examiner appears to be equating the claimed generation of the annotation 
data by combining phoneme data and words, to the generation of the phonetic index of Kupiec 
which is described at column 30, lines 38 to 44. However, Applicants submit that the phonetic 
index of Kupiec cannot be equated with the annotation data. Firstly, the phonetic index 
described by Kupiec is simply a dictionary of all possible words known to the system, together 
with a corresponding pronunciation (defined by a sequence of phonemes). The phonetic index in 
no way "annotates" the text in the database. Applicants submit that it cannot, therefore, be 
reasonably interpreted as being the claimed annotation data. 
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Secondly, in the system described by Kupiec, the phonemes generated by the 
speech recognizer are converted into words using the phonetic index. The interpretation of the 
phonetic index as being equivalent to the annotation data cannot, therefore, be possible, since this 
would be needed (to generate the words from the phonemes) before it is generated. 

In Section 75, it is indicated that the annotation graphs described by Bird have 
similarities to a network embodiment of the phonetic index described by Kupiec. However, the 
only similarity between the phonetic index described by Kupiec and the "annotation graphs" 
described by Bird is that they both include phoneme and word data. The two data structures are 
used for completely different purposes and have completely different structures. 

In the last paragraph of Section 75 of the December 13, 2002 Official Action, it is 
suggested that Bird discloses automatically generating the phonemes and words for the linguistic 
annotations that they discuss. However, Applicants submit that there is no such disclosure by 
Bird. As discussed above, although Bird discusses in Section 1 that there is "software" for 
creating the databases, Applicants submit that these software tools are for use by expert linguists 
and do not imply any automatic generation of the phoneme and words as suggested in the Office 
Action. 

Finally, with regard to the combination of Kupiec and Bird, Applicants submit 
that it would not have been obvious to one of ordinary skill in the art to make the combination 
suggested by the Examiner. In particular, Kupiec does not disclose or suggest generating any 
annotation data for a database. In other parts of the Official Action (see Section 65), the 
Examiner takes the position that Kupiec teaches generating the phonetic index through 
"automatic recognition". However, Applicants submit that it is clear, from column 21, lines 12 
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to 14 of Kupiec, that this automatic generation of the phonetic index is achieved using text-to- 
speech synthesis techniques and not speech recognition techniques. Kupiec is therefore silent on 
the possibility of generating annotation data (including the Examiner's interpretation of the 
phonetic index) using speech recognition. Applicants submit, therefore, that there is no 
motivation taught by Kupiec that the phoneme and word lattices proposed by Bird could be 
populated using speech recognition. 

Applicants therefore submit that the new claims are patentable over Bird and 
Kupiec, whether taken alone or in combination, at least for the feature of generating annotation 
data using automatic speech recognition as recited in the claims. 

Further, regarding the secondary references cited in the August 14, 2003 Office 
Action, Applicants believe that these references fail to alter this patentability. 

Favorable consideration and early passage to issue of the present application are 
respectfully requested. 
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Applicants' undersigned attorney may be reached in our Washington, D.C. office 



by telephone at (202) 530-1010. All correspondence should be directed to our address given 



FITZPATRICK, CELLA, HARPER & SCINTO 
30 Rockefeller Plaza 
New York, New York 101 12-3801 
Facsimile: (212)218-2200 
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below. 



Respectfully submitted, 




Attorney fy&r Applicants 
Melody H.Wu 
Registration No. 52,376 
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